Companies and organizations increasingly use audio/video conferencing and multipoint conferencing to improve communication and efficiency within the organization. Large organizations distribute a large number of multimedia endpoints throughout the organization. Usually, one or more multipoint control units (MCUs) serve the internal multipoint multimedia conferencing needs of these endpoints. Organizations that do not have private MCUs may purchase conferencing services from a multimedia conferencing service provider that has one or more MCUs.
A multimedia endpoint is a networked terminal that is capable of providing real-time, two-way audiovisual communication with other terminals or an MCU. An endpoint can provide speech only; speech and video; or speech, data and video communication. A MCU is a conference control entity located at a node of the network or in a terminal. The MCU receives multiple media channels from endpoints via access ports. According to certain criteria, the MCU processes audiovisual and data signals and distributes the processed signals to the connected channels. Examples of MCUs include the MGC-100 (Polycom Inc.). A more thorough definition of an endpoint (terminal) and an MCU can be found in the International Telecommunication Union (“ITU”) standards, such as but not limited to the H.320, H.324, and H.323 standards. Additional information regarding the ITU standards can be found at the ITU website.
Usually a MCU that is located in a terminal has limited functionality. For example, such an MCU might be capable of conducting a conference of only up to four participants. Model number VSX 8000 (Polycom Inc.) is an example for a terminal that includes an MCU.
Videoconferences can be conducted by two or more MCUs. One example is when the videoconference requires more resources than are available on a single MCU. In such a case, the conferees may be divided into two or more groups and each group may be associated with a different MCU. As another example, in a conference between conferees located in different sites, each site having its own MCU, each conferee may use his own local MCU and the entire conference may be conducted by cascading the different local MCUs. Those skilled are familiar with other examples of using cascades of MCUs to conduct videoconferences.
Some techniques have been disclosed for utilizing the resources of one or more MCUs, for cascading one or more MCUs, and generally for improving resource usage of one or more MCUs. See, for example, U.S. patent application Ser. Nos. 09/708,898 and 09/852,438; PCT international published patent applications WO02/060126 and WO2005/004481; U.S. Patent Application Publication No. 2003/0103468; and U.S. patent application Ser. No. 11/109,563, each of which are incorporated herein by reference.
In a common cascading conference, one of the MCUs can be defined as a master MCU (MMCU) and the other MCUs can be defined as slave MCUs (SMCUs). The MMCU has a multimedia connection to each one of the SMCUs. The multimedia connection between the MMCU and each one of the SMCUs can be similar to a multimedia connection with an endpoint of a conferee.
Usually a SMCU receives audio and video from conferees that are associated with the SMCU, mixes the audio, and handles the video of its associated conferees to create an associated videoconference. The video of the associated conference is organized according to the type of the cascading conference being conducted. One type of conference is a video switching conference. A video switching conference is a conference in which each conferee can see only one selected participant (one source of video). During the conference, the selected conferee can be changed according to different criteria and according to the dynamic of the conference. If the cascading conference is a video switching conference, the SMCU may select one of the associated conferee's video and uses it as the video of the associated conference of the SMCU.
One selection parameter for selecting which conferee is displayed at a given time during a video switching conference can be the audio energy of conferees. For example, the conferee with the highest audio energy might be displayed. Another selection parameter is a “forced presented conferee,” which is a conferee that has been defined to be presented in the video image of the conference independent of his activity. A conferee can be designated as a forced presented conferee by the moderator of the conference, an operator, as parameters in the conference profile, etc.
An alternative to video switching is a continuous presence layout. A SMCU may compose video signals of selected conferees from its associated group into a continuous presence (CP) video according to a layout of the associated conference of the SMCU. The mixed audio and the video of the associated conference are transferred in a manner similar to video and audio of a single conferee to the MMCU. A common continuous presence (“CP”) process involves scaling the video data from various source user terminals to change the frame resolution in order to incorporate it later into a continuous presence layout and video mixing, thus scaled video data to be composed in the CP layout.
The MMCU can mix the audio and video received from the one or more SMCUs with the audio and video of selected conferees from the group of conferees that are associated with the MMCU. The result is a mixed audio and video of the cascading conference. Then the MMCU may deliver the mixed audio and video of the cascading conference to each one of its own associated conferees and to the connected one or more SMCUs. Each one of the one or more SMCUs can distribute the mixed audio and video of the cascading conference to their associated conferee endpoints.
One challenge in managing a common cascading conference is that each MCU (MMCU and SMCUs) selects conferees to be mixed and displayed from its associated group independent of how the selected conferees relate to conferees in the other associated groups. Also, the sizes of the images of conferees that are associated with the MMCU are often different from the images of conferees that are associated with the SMCUs. If a CP layout is used as the layout of an associated conference of a SMCU, the composed video coming from the SMCU replaces a location of a single conferee in the cascading layout. The result is that each of the conferees associated with the SMCU gets a smaller screen area (e.g., ¼ of a space normally assigned to a conferee in the layout of 2×2), as is illustrated in FIG. 1 layout 100.
Layout 100 illustrates a 2×2 layout of CP cascading conference that is composed by a MMCU. In order to fit a conferee's window in a 2×2 layout, each selected video is scaled down to a quarter of a screen. The video images of selected conferees (AM, BM, DM) that are associated with the MMCU are scaled down and are placed in windows 102, 104, and 106, respectively. The input from the SMCU, which includes a 2×2 composite layout of the video images from conferees that are associated with the SMCU (As, Cs, Ds and Es), is also scaled down by four and is placed in the top right corner of layout 100. However, the image of each one of the slave conferees (As, Cs, Ds and Es) are scaled down to quarter of a screen by the SMCU, therefore, in the layout 100 of the cascading conference, each one of the slave conferees has a smaller area than one that is used by a conferee that is associated with the MMCU, i.e., 1/16 of the screen compare to a ¼ of a screen, as illustrated by rectangles 112, 114, 116 and 118.
A common way to correct the differences in sizes of images of conferees, which are associated with an MMCU compare to the image size of conferees that are associated with a SMCU, is by forcing the SMCU to use a video-switching layout and deliver video of a single selected conferee. The image of the single selected conferee is placed in the layout of the cascading conference. Layout 120 illustrates a snapshot of a screen of 2×2 CP cascading conference in which the SMCU is forced to work in switching mode. In switching mode, the SMCU delivers an image of a single selected conferee that covers the entire frame. Therefore, when the MMCU scales the image down to place it in the CP layout of the cascading video the scaled down image has the same size as the images of conferees that are associated with the MMCU. In layout 120, windows 122, 126, and 128 are associated with conferees AM, CM, DM (respectively) that are associated with the MMCU and the window 124 is associated with conferee as that is associated with the SMCU. The four areas (windows) (122, 126, 128 and 124) have the same size. Using this method corrects the size problem but prevents viewing the other conferees that are associated with the SMCU even if their audio energy is higher than the audio energy of AM and/or BM and/or DM.
Therefore, there is a need for systems and methods for composing video for cascading conference in which each of the conferees are evaluated under the same criteria for being displayed in the layout regardless of whether the conferee is associated with the MMCU or with a SMCU.